{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "images.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "private_outputs": true,
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    }
  },
  "cells": [
    {
      "metadata": {
        "colab_type": "text",
        "id": "mt9dL5dIir8X"
      },
      "cell_type": "markdown",
      "source": [
        "##### Copyright 2018 The TensorFlow Authors."
      ]
    },
    {
      "metadata": {
        "cellView": "form",
        "colab_type": "code",
        "id": "ufPx7EiCiqgR",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License.\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "ucMoYase6URl"
      },
      "cell_type": "markdown",
      "source": [
        "# tf.dataを使って画像をロードする"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "_Wwu5SXZmEkB"
      },
      "cell_type": "markdown",
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://www.tensorflow.org/tutorials/load_data/images\"><img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" />View on TensorFlow.org</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs/blob/master/site/ja/tutorials/load_data/images.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/tensorflow/docs/blob/master/site/ja/tutorials/load_data/images.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "metadata": {
        "id": "uybAm_B9LvF7",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Note: これらのドキュメントは私たちTensorFlowコミュニティが翻訳したものです。コミュニティによる 翻訳は**ベストエフォート**であるため、この翻訳が正確であることや[英語の公式ドキュメント](https://www.tensorflow.org/?hl=en)の 最新の状態を反映したものであることを保証することはできません。 この翻訳の品質を向上させるためのご意見をお持ちの方は、GitHubリポジトリ[tensorflow/docs](https://github.com/tensorflow/docs)にプルリクエストをお送りください。 コミュニティによる翻訳やレビューに参加していただける方は、 [docs-ja@tensorflow.org メーリングリスト](https://groups.google.com/a/tensorflow.org/forum/#!forum/docs-ja)にご連絡ください。"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "Oxw4WahM7DU9"
      },
      "cell_type": "markdown",
      "source": [
        "このチュートリアルでは、'tf.data'を使って画像データセットをロードする簡単な例を示します。\n",
        "\n",
        "このチュートリアルで使用するデータセットは、クラスごとに別々のディレクトリに別れた形で配布されています。"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "hoQQiZDB6URn"
      },
      "cell_type": "markdown",
      "source": [
        "## 設定"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "DHz3JONNEHlj",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from __future__ import absolute_import, division, print_function, unicode_literals\n",
        "\n",
        "import tensorflow as tf\n",
        "tf.enable_eager_execution()\n",
        "tf.__version__"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "KT6CcaqgQewg",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "AUTOTUNE = tf.data.experimental.AUTOTUNE"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "rxndJHNC8YPM"
      },
      "cell_type": "markdown",
      "source": [
        "## データセットのダウンロードと検査"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "wO0InzL66URu"
      },
      "cell_type": "markdown",
      "source": [
        "### 画像の取得\n",
        "\n",
        "訓練を始める前に、ネットワークに認識すべき新しいクラスを教えるために画像のセットが必要です。最初に使うためのクリエイティブ・コモンズでライセンスされた花の画像のアーカイブを作成してあります。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "rN-Pc6Zd6awg",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import pathlib\n",
        "data_root = tf.keras.utils.get_file('flower_photos','https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz', untar=True)\n",
        "data_root = pathlib.Path(data_root)\n",
        "print(data_root)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "rFkFK74oO--g"
      },
      "cell_type": "markdown",
      "source": [
        "218MBをダウンロードすると、花の画像のコピーが使えるようになっているはずです。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "7onR_lWE7Njj",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "for item in data_root.iterdir():\n",
        "    print(item)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "4yYX3ZRqGOuq",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import random\n",
        "all_image_paths = list(data_root.glob('*/*'))\n",
        "all_image_paths = [str(path) for path in all_image_paths]\n",
        "random.shuffle(all_image_paths)\n",
        "\n",
        "image_count = len(all_image_paths)\n",
        "image_count"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "t_BbYnLjbltQ",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "all_image_paths"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "vkM-IpB-6URx"
      },
      "cell_type": "markdown",
      "source": [
        "### 画像の検査\n",
        "\n",
        "扱っている画像について知るために、画像のいくつかを見てみましょう。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "wNGateQJ6UR1",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import os\n",
        "attributions = (data_root/\"LICENSE.txt\").open(encoding='utf-8').readlines()[4:]\n",
        "attributions = [line.split(' CC-BY') for line in attributions]\n",
        "attributions = dict(attributions)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "jgowG2xu88Io",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import IPython.display as display\n",
        "\n",
        "def caption_image(image_path):\n",
        "    image_rel = pathlib.Path(image_path).relative_to(data_root)\n",
        "    return \"Image (CC BY 2.0) \" + ' - '.join(attributions[str(image_rel)].split(' - ')[:-1])\n",
        "    "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "YIjLi-nX0txI",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "for n in range(3):\n",
        "    image_path = random.choice(all_image_paths)\n",
        "    display.display(display.Image(image_path))\n",
        "    print(caption_image(image_path))\n",
        "    print()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "OaNOr-co3WKk"
      },
      "cell_type": "markdown",
      "source": [
        "### 各画像のラベルの決定"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "-weOQpDw2Jnu"
      },
      "cell_type": "markdown",
      "source": [
        "ラベルを一覧してみます。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "ssUZ7Qh96UR3",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "label_names = sorted(item.name for item in data_root.glob('*/') if item.is_dir())\n",
        "label_names"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "9l_JEBql2OzS"
      },
      "cell_type": "markdown",
      "source": [
        "ラベルにインデックスを割り当てます。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "Y8pCV46CzPlp",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "label_to_index = dict((name, index) for index,name in enumerate(label_names))\n",
        "label_to_index"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "VkXsHg162T9F"
      },
      "cell_type": "markdown",
      "source": [
        "ファイルとラベルのインデックスの一覧を作成します。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "q62i1RBP4Q02",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "all_image_labels = [label_to_index[pathlib.Path(path).parent.name]\n",
        "                    for path in all_image_paths]\n",
        "\n",
        "print(\"First 10 labels indices: \", all_image_labels[:10])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "i5L09icm9iph"
      },
      "cell_type": "markdown",
      "source": [
        "### 画像の読み込みと整形"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "SbqqRUS79ooq"
      },
      "cell_type": "markdown",
      "source": [
        "TensorFlowには画像を読み込んで処理するために必要なツールが備わっています。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "jQZdySHvksOu",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "img_path = all_image_paths[0]\n",
        "img_path"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "2t2h2XCcmK1Y"
      },
      "cell_type": "markdown",
      "source": [
        "以下は生のデータです。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "LJfkyC_Qkt7A",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "img_raw = tf.read_file(img_path)\n",
        "print(repr(img_raw)[:100]+\"...\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "opN8AVc8mSbz"
      },
      "cell_type": "markdown",
      "source": [
        "画像のテンソルにデコードします。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "Tm0tdrlfk0Bb",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "img_tensor = tf.image.decode_image(img_raw)\n",
        "\n",
        "print(img_tensor.shape)\n",
        "print(img_tensor.dtype)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "3k-Of2Tfmbeq"
      },
      "cell_type": "markdown",
      "source": [
        "モデルに合わせてリサイズします。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "XFpz-3_vlJgp",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "img_final = tf.image.resize_images(img_tensor, [192, 192])\n",
        "img_final = img_final/255.0\n",
        "print(img_final.shape)\n",
        "print(img_final.numpy().min())\n",
        "print(img_final.numpy().max())\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "aCsAa4Psl4AQ"
      },
      "cell_type": "markdown",
      "source": [
        "このあと使用するために、簡単な関数にまとめます。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "HmUiZJNU73vA",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def preprocess_image(image):\n",
        "    image = tf.image.decode_jpeg(image, channels=3)\n",
        "    image = tf.image.resize_images(image, [192, 192])\n",
        "    image /= 255.0  # normalize to [0,1] range\n",
        "\n",
        "    return image"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "einETrJnO-em",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def load_and_preprocess_image(path):\n",
        "    image = tf.read_file(path)\n",
        "    return preprocess_image(image)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "3brWQcdtz78y",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "image_path = all_image_paths[0]\n",
        "label = all_image_labels[0]\n",
        "\n",
        "plt.imshow(load_and_preprocess_image(img_path))\n",
        "plt.grid(False)\n",
        "plt.xlabel(caption_image(img_path))\n",
        "plt.title(label_names[label].title())\n",
        "print()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "n2TCr1TQ8pA3"
      },
      "cell_type": "markdown",
      "source": [
        "## `tf.data.Dataset`の構築"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "6H9Z5Mq63nSH"
      },
      "cell_type": "markdown",
      "source": [
        "### 画像のデータセット"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "GN-s04s-6Luq"
      },
      "cell_type": "markdown",
      "source": [
        "`tf.data.Dataset`を構築する最も簡単な方法は、`from_tensor_slices`メソッドを使うことです。\n",
        "\n",
        "文字列の配列をスライスすると、文字列のデータセットが出来上がります。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "6oRPG3Jz3ie_",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "path_ds = tf.data.Dataset.from_tensor_slices(all_image_paths)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "uML4JeMmIAvO"
      },
      "cell_type": "markdown",
      "source": [
        "`output_shapes`と`output_types`という2つのフィールドが、データセット中の要素の中身を示しています。この場合には、バイナリ文字列というスカラーのセットです。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "mIsNflFbIK34",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "print('shape: ', repr(path_ds.output_shapes))\n",
        "print('type: ', path_ds.output_types)\n",
        "print()\n",
        "print(path_ds)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "ZjyGcM8OwBJ2"
      },
      "cell_type": "markdown",
      "source": [
        "`preprocess_image`をファイルパスのデータセットにマップすることで、画像を実行時にロードし整形する新しいデータセットを作成します。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "D1iba6f4khu-",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "image_ds = path_ds.map(load_and_preprocess_image, num_parallel_calls=AUTOTUNE)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "JLUPs2a-lEEJ",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "plt.figure(figsize=(8,8))\n",
        "for n,image in enumerate(image_ds.take(4)):\n",
        "    plt.subplot(2,2,n+1)\n",
        "    plt.imshow(image)\n",
        "    plt.grid(False)\n",
        "    plt.xticks([])\n",
        "    plt.yticks([])\n",
        "    plt.xlabel(caption_image(all_image_paths[n]))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "P6FNqPbxkbdx"
      },
      "cell_type": "markdown",
      "source": [
        "###  `(image, label)`のペアのデータセット"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "YgvrWLKG67-x"
      },
      "cell_type": "markdown",
      "source": [
        "同じ`from_tensor_slices`メソッドを使ってラベルのデータセットを作ることができます。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "AgBsAiV06udj",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "label_ds = tf.data.Dataset.from_tensor_slices(tf.cast(all_image_labels, tf.int64))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "HEsk5nN0vyeX",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "for label in label_ds.take(10):\n",
        "    print(label_names[label.numpy()])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "jHjgrEeTxyYz"
      },
      "cell_type": "markdown",
      "source": [
        "これらのデータセットは同じ順番なので、zipすることで`(image, label)`というペアのデータセットができます。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "AOEWNMdQwsbN",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "image_label_ds = tf.data.Dataset.zip((image_ds, label_ds))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "yA2F09SJLMuM"
      },
      "cell_type": "markdown",
      "source": [
        "新しいデータセットの`shapes`と`types`は、それぞれのフィールドを示すシェイプと型のタプルです。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "DuVYNinrLL-N",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "print('image shape: ', image_label_ds.output_shapes[0])\n",
        "print('label shape: ', image_label_ds.output_shapes[1])\n",
        "print('types: ', image_label_ds.output_types)\n",
        "print()\n",
        "print(image_label_ds)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "2WYMikoPWOQX"
      },
      "cell_type": "markdown",
      "source": [
        "注：`all_image_labels`や`all_image_paths`のような配列がある場合、`tf.data.dataset.Dataset.zip`メソッドの代わりとなるのは、配列のペアをスライスすることです。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "HOFwZI-2WhzV",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "ds = tf.data.Dataset.from_tensor_slices((all_image_paths, all_image_labels))\n",
        "\n",
        "# The tuples are unpacked into the positional arguments of the mapped function\n",
        "# タプルは展開され、マップ関数の位置引数に割り当てられます\n",
        "def load_and_preprocess_from_path_label(path, label):\n",
        "    return load_and_preprocess_image(path), label\n",
        "\n",
        "image_label_ds = ds.map(load_and_preprocess_from_path_label)\n",
        "image_label_ds"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "vYGCgJuR_9Qp"
      },
      "cell_type": "markdown",
      "source": [
        "### 基本的な訓練手法"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "wwZavzgsIytz"
      },
      "cell_type": "markdown",
      "source": [
        "このデータセットを使ってモデルの訓練を行うには、データが\n",
        "\n",
        "* よくシャッフルされ\n",
        "* バッチ化され\n",
        "* 限りなく繰り返され\n",
        "* バッチが出来るだけ早く利用できる\n",
        "\n",
        "ことが必要です。\n",
        "\n",
        "これらの特性は`tf.data`APIを使えば簡単に付け加えることができます。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "uZmZJx8ePw_5",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "BATCH_SIZE = 32\n",
        "\n",
        "# シャッフルバッファのサイズをデータセットと同じに設定することで、データが完全にシャッフルされる\n",
        "# ようにできます。\n",
        "ds = image_label_ds.shuffle(buffer_size=image_count)\n",
        "ds = ds.repeat()\n",
        "ds = ds.batch(BATCH_SIZE)\n",
        "# `prefetch`を使うことで、モデルの訓練中にバックグラウンドでデータセットがバッチを取得できます。\n",
        "ds = ds.prefetch(buffer_size=AUTOTUNE)\n",
        "ds"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "6JsM-xHiFCuW"
      },
      "cell_type": "markdown",
      "source": [
        "注意すべきことがいくつかあります。\n",
        "\n",
        "1. 順番が重要です。\n",
        "\n",
        "  * `.repeat`の前に`.shuffle`すると、エポックの境界を越えて要素がシャッフルされます。（他の要素がすべて出現する前に2回出現する要素があるかもしれません）\n",
        "  * `.batch`の後に`.shuffle`すると、バッチの順番がシャッフルされますが、要素がバッチを越えてシャッフルされることはありません。\n",
        "\n",
        "1. 完全なシャッフルのため、`buffer_size`をデータセットと同じサイズに設定しています。データセットのサイズ未満の場合、値が大きいほど良くランダム化されますが、より多くのメモリーを使用します。\n",
        "\n",
        "1. シャッフルバッファがいっぱいになってから要素が取り出されます。そのため、大きな`buffer_size`が`Dataset`を使い始める際の遅延の原因になります。\n",
        "\n",
        "1. シャッフルされたデータセットは、シャッフルバッファが完全に空になるまでデータセットが終わりであることを伝えません。`.repeat`によって`Dataset`が再起動されると、シャッフルバッファが一杯になるまでもう一つの待ち時間が発生します。\n",
        "\n",
        "最後の問題は、`tf.data.Dataset.apply`メソッドを、融合された`tf.data.experimental.shuffle_and_repeat`関数と組み合わせることで対処できます。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "Ocr6PybXNDoO",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "ds = image_label_ds.apply(\n",
        "    tf.data.experimental.shuffle_and_repeat(buffer_size=image_count))\n",
        "ds = ds.batch(BATCH_SIZE)\n",
        "ds = ds.prefetch(buffer_size=AUTOTUNE)\n",
        "ds"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "GBBZMSuAmQVL"
      },
      "cell_type": "markdown",
      "source": [
        "### データセットをモデルにつなぐ\n",
        "\n",
        "`tf.keras.applications`からMobileNet v2のコピーを取得します。\n",
        "\n",
        "これを簡単な転移学習のサンプルに使用します。\n",
        "\n",
        "MobileNetの重みを訓練不可に設定します。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "KbJrXn9omO_g",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "mobile_net = tf.keras.applications.MobileNetV2(input_shape=(192, 192, 3), include_top=False)\n",
        "mobile_net.trainable=False"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "Y7NVWiLF3Vbf"
      },
      "cell_type": "markdown",
      "source": [
        "このモデルは、入力が`[-1,1]`の範囲に正規化されていることを想定しています。\n",
        "\n",
        "```\n",
        "help(keras_applications.mobilenet_v2.preprocess_input)\n",
        "```\n",
        "\n",
        "<pre>\n",
        "...\n",
        "This function applies the \"Inception\" preprocessing which converts\n",
        "the RGB values from [0, 255] to [-1, 1] \n",
        "...\n",
        "</pre>"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "CboYya6LmdQI"
      },
      "cell_type": "markdown",
      "source": [
        "このため、データをMobileNetモデルに渡す前に、入力を`[0,1]`の範囲から`[-1,1]`の範囲に変換する必要があります。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "SNOkHUGv3FYq",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def change_range(image,label):\n",
        "    return 2*image-1, label\n",
        "\n",
        "keras_ds = ds.map(change_range)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "QDzZ3Nye5Rpv"
      },
      "cell_type": "markdown",
      "source": [
        "MobileNetは画像ごとに`6x6`の特徴量の空間を返します。\n",
        "\n",
        "バッチを１つ渡してみましょう。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "OzAhGkEK6WuE",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# シャッフルバッファがいっぱいになるまで、データセットは何秒かかかります。\n",
        "image_batch, label_batch = next(iter(keras_ds))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "LcFdiWpO5WbV",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "feature_map_batch = mobile_net(image_batch)\n",
        "print(feature_map_batch.shape)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "vrbjEvaC5XmU"
      },
      "cell_type": "markdown",
      "source": [
        "MobileNetをラップしたモデルを作り、出力層である`tf.keras.layers.Dense`の前に、`tf.keras.layers.GlobalAveragePooling2D`で空間の軸に沿って平均値を求めます。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "X0ooIU9fNjPJ",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "model = tf.keras.Sequential([\n",
        "    mobile_net,\n",
        "    tf.keras.layers.GlobalAveragePooling2D(),\n",
        "    tf.keras.layers.Dense(len(label_names))])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "foQYUJs97V4V"
      },
      "cell_type": "markdown",
      "source": [
        "期待したとおりの形状の出力が得られます。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "1nwYxvpj7ZEf",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "logit_batch = model(image_batch).numpy()\n",
        "\n",
        "print(\"min logit:\", logit_batch.min())\n",
        "print(\"max logit:\", logit_batch.max())\n",
        "print()\n",
        "\n",
        "print(\"Shape:\", logit_batch.shape)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "pFc4I_J2nNOJ"
      },
      "cell_type": "markdown",
      "source": [
        "訓練手法を記述するためにモデルをコンパイルします。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "ZWGqLEWYRNvv",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "model.compile(optimizer=tf.train.AdamOptimizer(), \n",
        "              loss=tf.keras.losses.sparse_categorical_crossentropy,\n",
        "              metrics=[\"accuracy\"])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "tF1mO6haBOSd"
      },
      "cell_type": "markdown",
      "source": [
        "訓練可能な変数は2つ、全結合層の`weights`と`bias`です。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "pPQ5yqyKBJMm",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "len(model.trainable_variables) "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "kug5Wg66UJjl",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "model.summary()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "f_glpYZ-nYC_"
      },
      "cell_type": "markdown",
      "source": [
        "モデルを訓練します。\n",
        "\n",
        "普通は、エポックごとの本当のステップ数を指定しますが、ここではデモの目的なので3ステップだけとします。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "AnXPRNWoTypI",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "steps_per_epoch=tf.ceil(len(all_image_paths)/BATCH_SIZE).numpy()\n",
        "steps_per_epoch"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "q_8sabaaSGAp",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "model.fit(ds, epochs=1, steps_per_epoch=3)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "UMVnoBcG_NlQ"
      },
      "cell_type": "markdown",
      "source": [
        "## 性能\n",
        "\n",
        "注：このセクションでは性能の向上に役立ちそうな簡単なトリックをいくつか紹介します。詳しくは、[Input Pipeline Performance](https://www.tensorflow.org/guide/performance/datasets)を参照してください。\n",
        "\n",
        "上記の単純なパイプラインは、エポックごとにそれぞれのファイルを一つずつ読み込みます。これは、CPUを使ったローカルでの訓練では問題になりませんが、GPUを使った訓練では十分ではなく、いかなる分散訓練でも使うべきではありません。"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "oNmQqgGhLWie"
      },
      "cell_type": "markdown",
      "source": [
        "調査のため、まず、データセットの性能をチェックする簡単な関数を定義します。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "_gFVe1rp_MYr",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import time\n",
        "\n",
        "def timeit(ds, batches=2*steps_per_epoch+1):\n",
        "    overall_start = time.time()\n",
        "    # タイマーをスタートする前に、パイプラインの初期化の（シャッフルバッファを埋める）ため、\n",
        "    # バッチを１つ取得します\n",
        "    it = iter(ds.take(batches+1))\n",
        "    next(it)\n",
        "\n",
        "    start = time.time()\n",
        "    for i,(images,labels) in enumerate(it):\n",
        "        if i%10 == 0:\n",
        "            print('.',end='')\n",
        "    print()\n",
        "    end = time.time()\n",
        "\n",
        "    duration = end-start\n",
        "    print(\"{} batches: {} s\".format(batches, duration))\n",
        "    print(\"{:0.5f} Images/s\".format(BATCH_SIZE*batches/duration))\n",
        "    print(\"Total time: {}s\".format(end-overall_start))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "TYiOr4vdLcNX"
      },
      "cell_type": "markdown",
      "source": [
        "現在のデータセットの性能は次のとおりです。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "ZDxLwMJOReVe",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "ds = image_label_ds.apply(\n",
        "    tf.data.experimental.shuffle_and_repeat(buffer_size=image_count))\n",
        "ds = ds.batch(BATCH_SIZE).prefetch(buffer_size=AUTOTUNE)\n",
        "ds"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "IjouTJadRxyp",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "timeit(ds)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "HsLlXMO7EWBR"
      },
      "cell_type": "markdown",
      "source": [
        "### キャッシュ"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "lV1NOn2zE2lR"
      },
      "cell_type": "markdown",
      "source": [
        "`tf.data.Dataset.cache`を使うと、エポックを越えて計算結果を簡単にキャッシュできます。特に、データがメモリに収まるときには効果的です。\n",
        "\n",
        "ここでは、画像が前処理（デコードとリサイズ）された後でキャッシュされます。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "qj_U09xpDvOg",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "ds = image_label_ds.cache()\n",
        "ds = ds.apply(\n",
        "    tf.data.experimental.shuffle_and_repeat(buffer_size=image_count))\n",
        "ds = ds.batch(BATCH_SIZE).prefetch(buffer_size=AUTOTUNE)\n",
        "ds"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "rdxpvQ7VEo3y",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "timeit(ds)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "usIv7MqqZQps"
      },
      "cell_type": "markdown",
      "source": [
        "メモリキャッシュを使う際の欠点のひとつは、実行の都度キャッシュを再構築しなければならないことです。このため、データセットがスタートするたびに同じだけ起動のための遅延が発生します。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "eKX6ergKb_xd",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "timeit(ds)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "jUzpG4lYNkN-"
      },
      "cell_type": "markdown",
      "source": [
        "データがメモリに収まらない場合には、キャッシュファイルを使用します。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "vIvF8K4GMq0g",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "ds = image_label_ds.cache(filename='./cache.tf-data')\n",
        "ds = ds.apply(\n",
        "    tf.data.experimental.shuffle_and_repeat(buffer_size=image_count))\n",
        "ds = ds.batch(BATCH_SIZE).prefetch(1)\n",
        "ds"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "eTIj6IOmM4yA",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "timeit(ds)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "qqo3dyB0Z4t2"
      },
      "cell_type": "markdown",
      "source": [
        "キャッシュファイルには、キャッシュを再構築することなくデータセットを再起動できるという利点もあります。2回めがどれほど早いか見てみましょう。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "hZhVdR8MbaUj",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "timeit(ds)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "WqOVlf8tFrDU"
      },
      "cell_type": "markdown",
      "source": [
        "### TFRecord ファイル"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "y1llOTwWFzmR"
      },
      "cell_type": "markdown",
      "source": [
        "#### 生の画像データ\n",
        "\n",
        "TFRecordファイルは、バイナリの大きなオブジェクトのシーケンスを保存するための単純なフォーマットです。複数のサンプルを同じファイルに詰め込むことで、TensorFlowは複数のサンプルを一度に読み込むことができます。これは、特にGCSのようなリモートストレージサービスを使用する際の性能にとって重要です。\n",
        "\n",
        "最初に、生の画像データからTFRecordファイルを構築します。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "EqtARqKuHQLu",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "image_ds = tf.data.Dataset.from_tensor_slices(all_image_paths).map(tf.read_file)\n",
        "tfrec = tf.data.experimental.TFRecordWriter('images.tfrec')\n",
        "tfrec.write(image_ds)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "flR2GXWFKcO1"
      },
      "cell_type": "markdown",
      "source": [
        "次に、TFRecordファイルを読み込み、以前定義した`preprocess_image`関数を使って画像のデコード/リフォーマットを行うデータセットを構築します。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "j9PVUL2SFufn",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "image_ds = tf.data.TFRecordDataset('images.tfrec').map(preprocess_image)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "cRp1eZDRKzyN"
      },
      "cell_type": "markdown",
      "source": [
        "これを、前に定義済みのラベルデータセットとzipし、期待通りの`(image,label)`のペアを得ます。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "7XI_nDU2KuhS",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "ds = tf.data.Dataset.zip((image_ds, label_ds))\n",
        "ds = ds.apply(\n",
        "    tf.data.experimental.shuffle_and_repeat(buffer_size=image_count))\n",
        "ds=ds.batch(BATCH_SIZE).prefetch(AUTOTUNE)\n",
        "ds"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "3ReSapoPK22E",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "timeit(ds)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "wb7VyoKNOMms"
      },
      "cell_type": "markdown",
      "source": [
        "これは、`cache`バージョンよりも低速です。前処理をキャッシュしていないからです。"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "NF9W-CTKkM-f"
      },
      "cell_type": "markdown",
      "source": [
        "#### シリアライズしたテンソル"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "J9HzljSPkxt0"
      },
      "cell_type": "markdown",
      "source": [
        "前処理をTFRecordファイルに保存するには、前やったように前処理した画像のデータセットを作ります。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "OzS0Azukkjyw",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "paths_ds = tf.data.Dataset.from_tensor_slices(all_image_paths)\n",
        "image_ds = paths_ds.map(load_and_preprocess_image)\n",
        "image_ds"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "onWOwLpYlzJQ"
      },
      "cell_type": "markdown",
      "source": [
        "`.jpeg`文字列のデータセットではなく、これはテンソルのデータセットです。\n",
        "\n",
        "これをTFRecordファイルにシリアライズするには、まず、テンソルのデータセットを文字列のデータセットに変換します。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "xxZSwnRllyf0",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "ds = image_ds.map(tf.serialize_tensor)\n",
        "ds"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "w9N6hJWAkKPC",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "tfrec = tf.data.experimental.TFRecordWriter('images.tfrec')\n",
        "tfrec.write(ds)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "OlFc9dJSmcx0"
      },
      "cell_type": "markdown",
      "source": [
        "前処理をキャッシュしたことにより、データはTFRecordファイルから非常に効率的にロードできます。テンソルを使用する前にデシリアライズすることを忘れないでください。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "BsqFyTBFmSCZ",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "RESTORE_TYPE = image_ds.output_types\n",
        "RESTORE_SHAPE = image_ds.output_shapes\n",
        "\n",
        "ds = tf.data.TFRecordDataset('images.tfrec')\n",
        "\n",
        "def parse(x):\n",
        "    result = tf.parse_tensor(x, out_type=RESTORE_TYPE)\n",
        "    result = tf.reshape(result, RESTORE_SHAPE)\n",
        "    return result\n",
        "\n",
        "ds = ds.map(parse, num_parallel_calls=AUTOTUNE)\n",
        "ds"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "OPs_sLV9pQg5"
      },
      "cell_type": "markdown",
      "source": [
        "次にラベルを追加し、以前と同じような標準的な処理を適用します。"
      ]
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "XYxBwaLYnGop",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "ds = tf.data.Dataset.zip((ds, label_ds))\n",
        "ds = ds.apply(\n",
        "  tf.data.experimental.shuffle_and_repeat(buffer_size=image_count))\n",
        "ds=ds.batch(BATCH_SIZE).prefetch(AUTOTUNE)\n",
        "ds"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "code",
        "id": "W8X6RmGan1-P",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "timeit(ds)"
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}